- 1 - 



SYMMETRIC TAGS 

FIELD OF INVENTION 

This invention relates to coded data for disposal on or in a surface and in particular to coded data formed 
5 from symmetric tags. In use the symmetric tags can be used for providing identity-coding and position- 
coding of surfaces. 

CROSS-REFERENCES 

Various methods, systems and apparatus relating to the present invention are disclosed in the following co- 
10 pending applications and granted patents filed by the applicant or assignee of the present invention. The 
disclosures of all of these co-pending applications are incorporated herein by cross-reference. 
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Some applications are temporarily identified by their docket number. This will be replaced by the 
corresponding application number when available. 

25 BACKGROUND ART 

The reference to any prior art in this specification is not, and should not be taken as, an acknowledgment or 
any form of suggestion that such prior art forms part of the common general knowledge. 

It is known to provide one or more coded data structures on a surface that can be read and decoded by a 
30 suitable sensing device. Various embodiments of such a device incorporating an optical sensor are described 
in many of the documents incorporated into the present application by cross-reference. 

The coded data structures disclosed in these documents include target features that enable the sensing device 
to identify the position of each structure. The relative positions of the features within each structure can also 
35 be interpreted to detennine perspective distortion of the structure as sensed, enabling perspective correction 
to be performed on the sensed data. However, to enable the sensing device to decode the data in the structure, 
it is necessary that the position and orientation of the data relative to the structure and the position and 
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orientation of the structure relative to the viewpoint be determined. Typically, this is achieved by providing 
at least one feature in the structure that is asymmetric in some way. For example, in one embodiment, a 
keyhole-shaped feature is provided that can be located with respect to the other features, and then recognized 
to ascertain the rotational orientation of the structure in relation to the sensing device. The actual data that is 
5 encoded in the data structure can then be decoded, since its position and orientation relative to the structure 
can be inferred. 

Disadvantages with this arrangement include the need to dedicate space to one or more orientation features, 
and the difficulty of including redundancy in such features for the purposes of allowing orientation 
10 determination in the presence of damage to the features. It is desirable, therefore, to encode orientation 
information both more space-efficiently and in an error-detectable and/or error-correctable fashion. 



SUMMARY OF THE INVENTION 

15 In a first aspect the present invention provides coded data for disposal on or in a surface, the coded data being 
arranged in accordance with at least one layout having n-fold rotational symmetry about a center of rotation, 
where n is at least two, and having mirror symmetry, the layout including: 

(a) n identical first sub-layouts rotated l/n revolutions apart about the center of rotation, and, 

(b) n identical second sub-layouts rotated l/n revolutions apart about the center of rotation, each second 
20 sub-layout being a reflection of a corresponding one of the first sub-layouts, at least one sub-layout 

including: 

(i) rotation-indicating data that distinguishes that sub-layout from at least one other sub- 
layout; and, 

(ii) reflection-indicating data that distinguishes the reflection of that sub-layout from the 
25 reflection of a corresponding reflected sub-layout. 

In another aspect the present invention provides a surface bearing machine-readable coded data, the coded 
data being arranged in accordance with at least one layout having n-fold rotational symmetry about a center 
of rotation, where n is at least two, and having mirror symmetry, the layout including: 
30 (c) n identical first sub-layouts rotated l/n revolutions apart about the center of rotation, and, 

(d) n identical second sub-layouts rotated l/n revolutions apart about the center of rotation, each second 
sub-layout being a reflection of a corresponding one of the first sub-layouts, at least one sub-layout 
including: 

(i) rotation-indicating data that distinguishes that sub-layout from at least one other sub- 
35 layout; and, 

(ii) reflection-indicating data that distinguishes the reflection of that sub-layout from the 
reflection of a corresponding reflected sub-layout. 
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In a further aspect the present invention provides a method of generating an interface surface, the method 
including the steps of: 

(e) receiving user data in a printer; 

(f) generating machine-readable coded data incorporating the user data, the coded data being arranged 
5 in accordance with at least one layout having n-fold rotational symmetry about a center of rotation, 

where n is at least two, and having mirror symmetry, the layout including: 

(i) n identical first sub-layouts rotated l/n revolutions apart about the center of rotation, and, 

(ii) n identical second sub-layouts rotated l/n revolutions apart about the center of rotation, 
each second sub-layout being a reflection of a corresponding one of the first sub-layouts, at 

1 0 least one sub-layout including: 

(1) rotation-indicating data that distinguishes that sub-layout from at least one other sub- 
layout; and, 

(2) reflection-indicating data that distinguishes the reflection of that sub-layout from the 
reflection of a corresponding reflected sub-layout. 

1 5 (g) printing the coded data onto a surface. 

In another aspect the present invention provides a method of using a sensing device to read machine-readable 
coded data, the coded data being arranged in accordance with at least one layout having /7-fold rotational 
symmetry about a center of rotation, where n is at least two, and having mirror symmetry, the layout 
including: 

(i) n identical first sub-layouts rotated l/n revolutions apart about the center of rotation, and, 

(ii) n identical second sub-layouts rotated l/n revolutions apart about the center of rotation, 
each second sub-layout being a reflection of a corresponding one of the first sub-layouts, at 
least one sub-layout including: 

(1) rotation-indicating data that distinguishes that sub-layout from at least one other sub- 
layout; and, 

(2) reflection-indicating data that distinguishes the reflection of that sub-layout from the 
reflection of a corresponding reflected sub-layout. 

(h) sensing at least one layout using the sensing device; 

(i) decoding the coded data of at least one of the sub-layouts of the sensed layout, thereby determining 
at least the rotation-indicating data of that sub-layout; and 

(j) using the rotation-indicating data to interpret the meaning of at least some of the coded data. 

BRIEF DESCRIPTION OF THE DRAWINGS 

Preferred and other embodiments of the invention will now be described, by way of non-limiting example 
35 only, with reference to the accompanying drawings, in which: 

Figure 1 shows a triangular macrodot packing with a four-bit symbol unit outlined, for use with an 
embodiment of the invention; 
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Figure 2 shows a square macrodot packing with a four-bit symbol unit outlined, for use with an embodiment 
of the invention; 

Figure 3 shows a hexagonal macrodot packing with a four-bit symbol unit outlined, for use with an 
embodiment of the invention; 
5 Figure 4 shows a one-sixth segment of an hexagonal tag, with the segment containing a maximum of 1 1 four- 
bit symbols with the triangular macrodot packing shown in Figure 1 ; 

Figure 5 shows a one-sixth segment of another hexagonal tag, with the segment containing a maximum of 17 
four-bit symbols with the triangular macrodot packing shown in Figure 1 ; 

Figure 6 shows a one-quarter segment of a square tag, with the segment containing a maximum of 15 four-bit 
1 0 symbols with the square macrodot packing shown in Figure 2; 

Figure 7 shows a one-sixth segment of an hexagonal tag, with the segment containing a maximum of 14 four- 
bit symbols with the hexagonal macrodot packing shown in Figure 3; 

Figure 8 shows a logical layout of a hexagonal tag using the tag segment of Figure 4, with six interleaved 2 4 - 
ary codewords; 

1 5 Figure 9 shows the macrodot layout of the hexagonal tag of Figure 8; 

Figure 10 shows an arrangement of seven abutting tags of the design of Figure 8 and Figure 9, with shared 
targets; 

Figure 1 1 shows a logical layout of a square tag using the tag segment of Figure 6, with four interleaved 2 4 - 
ary (15, k) codewords; 
20 Figure 12 shows the macrodot layout of the square tag of Figure 11; 

Figure 13 shows an arrangement of nine abutting tags of the design of Figure 1 1 and Figure 12, with shared 
targets; 

Figure 14 shows a logical layout of an alternative square tag, with four interleaved 2 3 -ary (7,Ar) codewords; 
Figure 15 shows the macrodot layout of the square tag of Figure 14; 
25 Figure 16 shows a logical layout of a triangular tag, with three interleaved 2 3 -ary (7, k) codewords; 
Figure 17 shows the macrodot layout of the triangular tag of Figure 16; 

Figure 18 shows a logical layout of a two-fold rotationally symmetric linear tag, with two interleaved 2 3 -ary 
(7,k) codewords; 

Figure 19 shows the macrodot layout of the linear tag of Figure 18; 
30 Figure 20 shows an alternative macrodot layout based on the layout of the linear tag of Figure 18, with a 
purely one-dimensional structure analogous to the structure of a conventional barcode; 

Figure 21 shows a logical layout of an alternative hexagonal tag, with one local 2 4 -ary (12, k) codeword, 
interleaved with eighteen 3-symbol fragments of eighteen distributed 2 4 -ary (9, k) codewords; 
Figure 22 shows the logical layout of the hexagonal tag of Figure 21, re-arranged to show the distributed 3- 
35 symbol fragments which contribute to the same codewords; 

Figure 23 shows a continuous tiling of three hexagonal tag types each with the structure of the tag of Figure 
21; 

Figure 24 shows a layout of an alternative square tag using the symbol of Figure 2, with eight interleaved 2 4 - 
ary (6, k) codewords; 
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Figure 25 shows a layout of an alternative hexagonal tag using the symbol of Figure 1, with twelve 
interleaved 2 4 -ary (4,£) codewords; and 

Figure 26 shows a tag image processing and decoding process flow. 

5 DESCRIPTION OF THE PREFERRRED AND OTHER EMBODIMENTS 

In the preferred embodiment, the invention is configured to work with the netpage networked computer 
system, an overview of which follows. It will be appreciated that not every implementation will necessarily 
embody all or even most of the specific details and extensions discussed in relation to the basic netpage 
system. 

10 

In its preferred form, the netpage system relies on the production of, and human interaction with, netpages. 
These are pages of text, graphics and images printed on ordinary paper, but which work like interactive web 
pages. Information is encoded on each page using ink which is substantially invisible to the unaided human 
eye. The ink, however, and thereby the coded data, can be sensed by an optically imaging pen and 
1 5 transmitted to the netpage system. 

In the preferred form, active buttons and hyperlinks on each page can be clicked with a sensing pen to 
request information from the network or to signal preferences to a network server. In one embodiment, text 
written by hand on a netpage is automatically recognized and converted to computer text in the netpage 
20 system, allowing forms to be filled in. In other embodiments, signatures recorded on a netpage are 
automatically verified, allowing e-commerce transactions to be securely authorized. 

A detailed description of several forms of the netpage system, as well as a netpage printer and a netpage 
sensing device in the form of a stylus and pen is described in the present applicants' co-pending PCT 
25 application WO 00/72230 entitled "Sensing Device, filed 24 May 2000; and co-pending US application 
USSN 09/721,893 entitled "Sensing Device", filed 25 November 2000. 

1.1 NETPAGES 

Netpages are the foundation on which a netpage network is built. They provide a paper-based user interface 
30 to published information and interactive services. 

A netpage consists of a printed page (or other surface region) invisibly tagged with references to an online 
description of the page. The online page description is maintained persistently by a netpage page server. The 
page description describes the visible layout and content of the page, including text, graphics and images. It 
35 also describes the input elements on the page, including buttons, hyperlinks, and input fields. A netpage 
allows markings made with a netpage pen on its surface to be simultaneously captured and processed by the 
netpage system. 
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Multiple netpages can share the same page description. However, to allow input through otherwise identical 
pages to be distinguished, each netpage is assigned a unique page identifier. This page ID has sufficient 
precision to distinguish between a very large number of netpages. 

5 

Each reference to the page description is encoded in a printed tag. The tag identifies the unique page on 
which it appears, through the page ID, and thereby indirectly identifies the page description. The tag also 
identifies its own position on the page. Characteristics of the tags are described in more detail below, 

10 Tags are printed in infrared-absorptive ink on any substrate which is infrared-reflective, such as ordinary 
paper. Near-infrared wavelengths are invisible to the human eye but are easily sensed by a solid-state image 
sensor with an appropriate filter. 

A tag is sensed by an area image sensor in the netpage pen, and the decoded tag data is transmitted to the 
1 5 netpage system via the nearest netpage relay, such as a suitably- enabled personal computer, PDA, mobile 
phone or netpage printer. The pen is wireless and communicates with the netpage relay via a short-range 
radio link. The pen may also incorporate the relay functionality. Tags are sufficiently small and densely 
arranged that the pen can reliably image at least one tag even on a single click on the page. It is important 
that the pen recognize the page ID and position on every interaction with the page, since the interaction is 
20 stateless. 

As only single tags may be sensed with one interaction, tags are preferably error-correctably encoded to 
make them partially tolerant to surface damage. 

25 The netpage page server maintains a unique page instance for each printed netpage, allowing it to maintain a 
distinct set of user-supplied values for input fields in the page description for each printed netpage. 

1.2 CODED DATA ON SURFACES USING NETPAGE TAGS 

Various netpage coding schemes and patterns are described in the present applicants' co-pending PCT 
30 application WO 00/72249 entitled "Identity-Coded Surface with Reference Points", filed 24 May 2000; co- 
pending PCT application WO 02/84473 entitled "Cyclic Position Codes", filed 1 1 October 2001; co-pending 
US application USSN 10/309,358 entitled "Rotationally Symmetric Tags", filed 4 December 2002; and 
Australian Provisional Application 2002952259 entitled "Methods and Apparatus (NPT019)", filed 25 
October 2002. 

35 
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1.2.1 Tag Data Content 

In a preferred form, each tag identifies the region in which it appears, and the location of that tag within the 
region. A tag may also contain flags which relate to the region as a whole or to the tag. One or more flag bits 
may, for example, signal a tag sensing device to provide feedback indicative of a function associated with the 
5 immediate area of the tag, without the sensing device having to refer to a description of the region. A netpage 
pen may, for example, illuminate an "active area" LED when in the zone of a hyperlink. 

The tags preferably tile the entire page, and are sufficiently small and densely arranged that the pen can 
reliably image at least one tag even on a single click on the page. It is important that the pen recognize the 
1 0 page ID and position on every interaction with the page, since the interaction is stateless. 

In a preferred embodiment, the region to which a tag refers coincides with an entire page, and the region ID 
encoded in the tag is therefore synonymous with the page ED of the page on which the tag appears. In other 
embodiments, the region to which a tag refers can be an arbitrary subregion of a page or other surface. For 
1 5 example, it can coincide with the zone of an interactive element, in which case the region ID can directly 
identify the interactive element. 

Each tag typically contains 16 bits of tag ID, at least 90 bits of region ID, and a number of flag bits. 
Assuming a maximum tag density of 64 per square inch, a 16-bit tag ID supports a region size of up to 1024 

20 square inches. Larger regions can be mapped continuously without increasing the tag ED precision simply by 
using abutting regions and maps. The distinction between a region ID and a tag ID is mostly one of 
convenience. For most purposes the concatenation of the two can be considered as a globally unique tag ID. 
Conversely, it may also be convenient to introduce structure into the tag ID, for example to define the x and y 
coordinates of the tag. A 90-bit region ID allows 2 90 (-10 27 or a thousand trillion trillion) different regions to 

25 be uniquely identified. Tags may also contain type information, and a region may be tagged with a mixture of 
tag types. For example, a region may be tagged with one set of tags encoding x coordinates and another set, 
interleaved with the first, encoding y coordinates. It will be appreciated the region tag ED and tag ID 
precision may be more or less than just described depending on the environment in which the system will be 
used. 

30 

1.2.2 Tag Data Encoding 

Figure 1 1 shows one embodiment of a netpage tag in the form of a square tag 726 with four perspective 
targets 17. It is similar in appearance to tags described by Bennett et al. in US Patent 5051746. The tag 726 
represents sixty 4-bit Reed-Solomon symbols 747 (see the description of Figure 1, Figure 2 and Figure 3 
35 below for a discussion of symbols), for a total of 240 bits. The tag represents each "one" bit by the presence 
of a mark 748, referred to as a macrodot, and each "zero" bit by the absence of the corresponding macrodot. 
Figure 13 shows a square tiling 728 of nine tags, containing all "one" bits for illustrative purposes. It will be 
noted that the perspective targets are designed to be shared between adjacent tags. 
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In general, each tag includes a codeword encoded four times within the tag using the 4-bit symbols. Using a 
(15, 7) Reed-Solomon code, 112 bits of tag data are redundantly encoded to produce 240 encoded bits, 
corresponding to 60-bits, or 15 4-bit symbols, per codeword. 

5 

In one example, each 4-bit symbol is represented in a spatially coherent way in the tag, with the symbols of 
the four codewords being interleaved spatially within the tag. This is shown by the example in Figure 11, 
where each symbol is labeled with the number of its codeword (1-4) and the position of the symbol within 
the codeword(A-O). 

10 

This ensures that a burst error (an error affecting multiple spatially adjacent bits) damages a minimum 
number of symbols overall and a minimum number of symbols in any one codeword, thus maximizing the 
likelihood that the burst error can be fully corrected. 

15 This arrangement of using a (15, 7) Reed-Solomon code, with 112 bits of tag data redundantly encoded to 
produce 240 encoded bits, and spatial interleaving of the codewords within the tag allows up to 4 symbol 
errors to be corrected per codeword, i.e. it is tolerant of a symbol error rate of up to 27% per codeword. 

Any suitable error-correcting code can be used in place of a (15, 7) Reed-Solomon code, for example: a 
20 Reed-Solomon code with more or less redundancy, with the same or different symbol and codeword sizes; 
another block code; or a different kind of code, such as a convolutional code (see, for example, Stephen B. 
Wicker, Error Control Systems for Digital Communication and Storage, Prentice-Hall 1995, the contents of 
which a herein incorporated by reference thereto). 

25 In order to support "single-click" interaction with a tagged region via a sensing device, the sensing device 
must be able to see at least one entire tag in its field of view no matter where in the region or at what 
orientation it is positioned. The required diameter of the field of view of the sensing device is therefore a 
function of the size and spacing of the tags. 

30 The data-bearing macrodots 748 of the tag are designed to not overlap their neighbors, so that groups of tags 
cannot produce structures that resemble targets. This also saves ink. The perspective targets allow detection 
of the tag, so further targets are not required. 

Although the tag may contain an orientation feature to allow disambiguation of the four possible orientations 
35 of the tag relative to the sensor, the present invention is concerned with embedding orientation and related 
transform data in the tag data. For example, the four codewords can be arranged so that each tag orientation 
(in a rotational sense) contains one codeword placed at that orientation, as shown in Figure 11. 

NPT037US 



- 10- 



Tag decoding then consists of decoding one codeword at each rotational orientation. Each codeword can 
either contain a single bit indicating whether it is the first codeword, or two bits indicating which codeword it 
is. The latter approach has the advantage that if, say, the data content of only one codeword is required, then 
at most two codewords need to be decoded to obtain the desired data. This may be the case if the region ID is 
5 not expected to change within a stroke and is thus only decoded at the start of a stroke. Within a stroke only 
the codeword containing the tag ID is then desired. Furthermore, since the rotation of the sensing device 
changes slowly and predictably within a stroke, only one codeword typically needs to be decoded per frame. 

It is possible to dispense with perspective targets altogether and instead rely on the data representation being 
1 0 self-registering. In this case each bit value (or multi-bit value) is typically represented by an explicit glyph, 
i.e. no bit value is represented by the absence of a glyph. This ensures that the data grid is well-populated, 
and thus allows the grid to be reliably identified and its perspective distortion detected and subsequently 
corrected during data sampling. 

15 To allow tag boundaries to be detected, each tag data must contain a marker pattern, and these must be 
redundantly encoded to allow reliable detection. The overhead of such marker patterns is similar to the 
overhead of explicit perspective targets. Such scheme uses dots positioned at various points relative to grid 
vertices to represent different glyphs and hence different multi-bit values, as discussed in the present 
applicants' co-pending PCT application WO 02/084473 entitled "Cyclic Position Codes", filed 1 1 October 

20 2001. 

The arrangement 728 of Figure 13 shows that the square tag 726 can be used to fully tile or tessellate, i.e. 
without gaps or overlap, a plane of arbitrary size, 

25 Although in preferred embodiments the tagging schemes described herein encode a single data bit using the 
presence or absence of a single undifferentiated macrodot, they can also use sets of differentiated glyphs to 
represent single-bit or multi-bit values, such as the sets of glyphs illustrated in the present applicants' co- 
pending PCT application WO 02/084473 entitled "Cyclic Position Codes", filed 1 1 October 2001. 

30 Macrodot Packing Schemes 

Figure 1 shows a triangular macrodot packing 700 with a four-bit symbol unit 702 outlined. The area of the 

symbol unit is given by A UNIT = 3.5s 2 , where s the spacing of adjacent macrodots. Figure 2 

shows a square macrodot packing 704 with a four-bit symbol unit 706 outlined. The area of the symbol unit 
is given by Aumt = 4s 2 . Figure 3 shows a hexagonal macrodot packing 708 with a four-bit symbol unit 710 

35 outlined. The area of the symbol unit is given by Aumt = 3 <y[~ 3s 2 = 5.2s 2 . Of these packing schemes, the 
triangular packing scheme gives the greatest macrodot density for a particular macrodot spacing s. 
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In preferred embodiments, s has a value between 100 nm and 200 fim. 
Tap Desig ns 

Figure 4 shows a one-sixth segment 712 of a hexagonal tag, with the segment containing a maximum of 1 1 
5 four-bit symbols with the triangular macrodot packing shown in Figure 1. The target 17 is .shared with 
adjacent segments. Each tag segment can, by way of example, support a codeword of an (11, &) Reed- 
Solomon code, i.e. a punctured (15, fc) code, with the ability to detect w = 1 l-k symbol errors, or correct / = 
|_(1 1 — k) 1 2 J symbol errors. For example, if k = 7 then u = 4 and t=2. 

10 (EQ 1) gives the area of the corresponding hexagonal tag. (EQ 2) gives the radius of the required field of 
view. (EQ 3) gives the area of the required field of view. (EQ 4) gives the recovered bit data density for the 
field of view. 



Atac = (10s) x (5 7~ 3s) x 6 = 300 3s 2 = 520s 2 (EQ 1) 

1 5 R FOV = (10s x 2) + 2s = 22s (EQ 2) 

A FO y = n R 2 FO v = 152 Is 2 (EQ 3) 

D FO v = (6 x 1 1 x 4)/A F0V = OM/s 2 (EQ 4) 



Figure 5 shows a one-sixth segment 716 of another hexagonal tag, with the segment containing a maximum 
20 of 17 four-bit symbols with the triangular macrodot packing shown in Figure 1. Each tag segment can, by 
way of example, support a codeword of a (17, k) Reed-Solomon code, i.e. an augmented (15, A:) code, with 
the ability to detect u = \l-k symbol errors, or correct t = [(17 - k)/2] symbol errors. For example, if k = 7 
then u = 10 and t = 5. 

25 (EQ 5) gives the area of the corresponding hexagonal tag. (EQ 6) gives the radius of the required field of 
view. (EQ 7) gives the area of the required field of view. (EQ 8) gives the recovered bit data density for the 
field of view. 



Atag = (12s) x (6 yf 3s) x 6 = 432 yj~ 3s 2 = 748s (EQ 5) 

30 R FOV = (12s jc 2) + 2s = 26s (EQ 6) 

^R 2 FO v = 2124s 2 (EQ7) 

Dfov = (6 x 17 x 4)/Afov = 0. 19/s 2 (EQ 8) 



Figure 6 shows a one-quarter segment 718 of a square tag, with the segment containing a maximum of 15 
35 four-bit symbols with the square macrodot packing shown in Figure 2. Each tag segment can, by way of 
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example, support a codeword of a (15, k) Reed-Solomon code, with the ability to detect u = 15 - k symbol 
errors, or correct t = |_( 1 5 - k ) / 2 J symbol errors. For example, if k = 7 then u = 8 and / = 4. 

(EQ 9) gives the area of the corresponding square tag. (EQ 10) gives the radius of the required field of view. 
5 (EQ 1 1) gives the area of the required field of view. (EQ 12) gives the recovered bit data density for the field 
of view. 



A T ag = (8s) x (Ss) x 4 = 256s 2 (EQ 9) 

Rfov = ((8s x 2) + 2s) x yj~ 2 = 25.5s (EQ 10) 

10 7TR 2 fov = 2036s 2 (EQ11) 

Dfov = (4 x 15 x 4)/A FO v = 0.1 2/s 2 (EQ 12) 



Figure 7 shows a one-sixth segment 720 of an hexagonal tag, with the segment containing a maximum of 14 
four-bit symbols with the hexagonal macrodot packing shown in Figure 3. Each tag segment can, by way of 
15 example, support a codeword of a (14, k) Reed-Solomon code, i.e. a punctured (15, k) code, with the ability 
to detect u = 14 - k symbol errors, or correct * = |_( 14 - k) / 2 J symbol errors. For example, if k = 6 then 
and u = 8 and * = 4. 

(EQ 13) gives the area of the hexagonal tag. (EQ 14) gives the radius of the required field of view. (EQ 15) 
20 gives the area of the required field of view. (EQ 16) gives the recovered bit data density for the field of view. 

Atag = (8 V~ 3s) x (1 2s) x 6 = 576 3s 2 = 998s 2 (EQ 13) 

Rfov = (8 <yf 3s x 2) + 2s = 29.7s (EQ 14) 

71 R 2 FOV = 2774s 2 (EQ 15) 

25 D FOV = (6 x 14 x 4)/A FOV = 0. 12/s 2 (EQ 16) 

Of the tag designs considered, hexagonal tag designs using the tag segments shown in Figure 4 and Figure 5 
yield fields of view with the highest recovered bit data densities. Square and hexagonal tag designs using the 
tag segments shown respectively in Figure 6 and Figure 7 yield fields of view with lower recovered bit data 
30 densities. 

Hexagonal Tag Design 

Figure 8 shows a logical layout of a hexagonal tag 722 using the tag segment 712 of Figure 4, with six 
interleaved 2 4 -ary (11, k) codewords. Figure 9 shows the macrodot layout of the hexagonal tag 722 of 
35 Figure 8. Figure 10 shows an arrangement 724 of seven abutting tags 722 of the design of Figure 9, with 
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shared targets 17. The arrangement 724 shows that the hexagonal tag 722 can be used to tessellate a plane of 
arbitrary size. 

Alternative Square Tag Design 
5 Figure 14 shows a logical layout of another square tag 730, with four interleaved 2 3 -ary (7, k) codewords. 
Each 3-bit symbol 732 is represented by an L-shaped arrangement of three macrodots. Figure 15 shows the 
macrodot layout of the square tag 730 of Figure 14. 

Triangular Tag Design 

10 Figure 16 shows a logical layout of a triangular tag 734, with three interleaved 2 3 -ary (7, k) codewords. Each 
3-bit symbol 736 is represented by a triangular arrangement of three macrodots. Figure 17 shows the 
macrodot layout of the triangular tag 734 of Figure 16. As with the square and hexagonal tags described 
above, the triangular tag 734 can be used to tessellate a plane of arbitrary size. 

15 Linear Tag Design 

Figure 18 shows a logical layout of a two- fold rotationally symmetric linear tag 738, with two interleaved 2 3 
-ary (7, k) codewords. Each 3-bit symbol 740 is represented by a linear arrangement of three macrodots. 
Figure 19 shows the macrodot layout of the linear tag 738 of Figure 18. 

20 Figure 20 shows an alternative macrodot layout based on the layout of the linear tag of Figure 18, with a 
purely one-dimensional structure analogous to the structure of a conventional barcode. Although the layout is 
based on that of the linear tag of Figure 18, the target structures 17 are linear rather than circular. This 
embodiment is particularly suitable for use with prior art barcode scanners (with suitable software changes) 
adapted for linear scanning rather than area scanning. 

25 

Distributed Hexagonal Tag Design 

Figure 21 shows a logical layout of a hexagonal tag 750 using the tag segment of Figure 22, with one local 2 4 
-ary (12, k) codeword interleaved with eighteen 3-symbol fragments of eighteen distributed 2 4 -ary (9, k) 
codewords. Figure 9 shows, as before, the macrodot layout of the hexagonal tag 750 of Figure 2 1 , and Figure 
30 10 shows an arrangement of seven abutting tags of the design of Figure 9, with shared targets. 

In the layout of Figure 21, the twelve 4-bit symbols of the local codeword are labeled Gl through G12, and 
are shown with a dashed outline. Each symbol of the eighteen fragments of the eighteen distributed 
codewords is labeled with an initial prefix of A through F, indicating which of six nominal codewords the 
35 symbol belongs to, a subsequent prefix of S through U, indicating which 3-symbol part of the codeword the 
symbol belongs to, and a suffix of 1 through 3, indicating which of the three possible symbols the symbol is. 
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Figure 23 shows a continuous tiling 752 of three hexagonal tag types labeled P, Q and R, each with the 
structure of tag 750. There are six distributed codewords (A through F) associated with each tag type (P, Q 
and R). The distributed tagging scheme is predicated on the data content of each respective distributed 
5 codeword AP, AQ 5 AR, etc. through FP, FQ and FR being constant throughout the tiling. Only the G 
codewords can contain unique data. 

Figure 23 shows lattice vectors 754 associated with the physical tag layout, and corresponding (i.e. iso- 
directional rather than minimal) lattice vectors 755 associated with the PQR tag layout. The ratio of the 
10 length of the PQR lattice vectors 755 to the length of the corresponding physical lattice vectors 754 is three, 
consistent with there being three distinct tag types P, Q and R in the PQR tag layout. 

Figure 23 also shows the coverage 756 and 757 of a minimal imaging field of view in two arbitrary positions 
with respect to the tiling. The field of view is minim al with respect to the tiling in the sense that it the 

1 5 smallest field of view which is guaranteed to include at least one complete tag. Tag 750 is structured so that 
the rninimal field of view allows the recovery of the local codeword G of at least one tag, and the entire set of 
distributed codewords AP through FR via fragments of tags of type P, Q and R included in the field of view. 
Furthermore, the continuous tiling of tag 750 ensures that there is a codeword available with a known layout 
for each possible rotational and translational combination (of which there are eighteen). Each distributed 

20 codeword includes data which identifies the rotation of the codeword in relation to the tiling, thus allowing 
the rotation of the tiling with respect to the field of view to be determined from decoded data rather than from 
other structures, and the local codeword to be decoded at the correct orientation. 

Once the field of view is imaged, decoding proceeds as follows. A codeword is sampled at an arbitrary 
25 rotation (of the possible six rotations with respect to the six targets). This consists of sampling three S 
symbols from the current tag, and sampling, at the same rotation, T and U symbols from adjacent tags, 
modulo the size and alignment of the field of view with respect to the tiling. The result is a single distributed 
2 4 -ary (9, k) codeword. Decoding the codeword yields, among other data, the codeword orientation. Once 
the orientation is known, the local G codeword can be sampled and decoded at the correct orientation. 
30 Decoding the G codeword yields, among other data, the type of the corresponding tag (i.e. P, Q or R). This in 
turn allows the data content of the various distributed codewords to be correctly interpreted, once they are 
sampled and decoded. It also allows the distributed codewords to be selectively sampled and decoded. For 
example, codewords known to contain data which is assumed not to change during a stroke, such as the 
region ID, may be ignored. 

35 

Figure 22 shows the logical layout of the hexagonal tag 750 of Figure 21, re-arranged to show the distributed 
3 -symbol fragments which contribute to the same codewords. For example, if the central tag shown in Figure 
22 were a P-type tag, then the six distributed codewords shown in the figure would be the AP, BP, CP, DP, 
EP and FP codewords. Figure 22 also shows the local G codeword of the tag. Clearly, given the distributed 
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and repeating nature of the distributed codewords, different fragments from the ones shown in the figure can 
be used to build the corresponding codewords. 

Although it is more space-efficient to record the tag type (P, Q or R) in the G codeword, the tag type 
5 corresponding to each (or to selected) distributed codewords can also be recorded in the distributed 
codewords themselves. In this case there is a fixed relationship between a codeword's alignment and the tag 
to which its tag type refers. For example, a codeword's tag type may always refer to the tag containing the S 
fragment of the codeword. 

10 Note that with a larger field of view, a particular codeword need not be constant throughout the tiling, but 
may instead be allowed to be constant only in a certain direction. For example, a codeword used to encode a 
coordinate can be constant in the direction where the coordinate is constant, but can vary in (semi- 
orthogonal directions (with respect to the rotational symmetry of the tiling). This has the advantage that 
semi-variable information can be encoded in distributed codewords rather than being restricted to local 

1 5 codewords. 



Mirror-Symmetric Tag Designs 

Figure 24 shows a layout of a square tag 760 using the symbol of Figure 2, with eight interleaved 2 4 -ary (6, 
k) codewords. Like the square tag of Figure 1 1 discussed earlier, the tag of Figure 24 contains four 
20 codewords, each arranged according to the same layout but each rotated according to a different one of the 
four possible rotations of the tag with respect to its own four- fold rotational symmetry. As discussed earlier, 
this allows a codeword to be sampled and decoded independendy of the actual rotation of the tag, which 
allows the rotation of the tag to be encoded in the codeword(s). 

25 The tag of Figure 24 contains four additional codewords, laid out in the mirror image of the original four 
codewords with respect to the tag's two axes of reflection 762 and 764. This allows a codeword to be 
sampled and decoded independently of the actual rotation and reflection of the tag, which allows the rotation 
and reflection of the tag to be encoded in the codeword(s). This is useful if the tag may be imaged in 
reflection, e.g. reflected in a mirror or through the back of a transparent substrate. 

30 

Each codeword may explicitly indicate its rotation and reflection via a three-bit value, or implicitly via a one- 
bit value indicating whether the codeword corresponds to a pre-determined rotation and reflection or not. 

In Figure 24, each original symbol has a prefix (1-4) indicating which codeword it belongs to, and a suffix 
35 (A-F) indicating its position in the codeword. Each reflected symbol is shown with a bar above its label, as 
well as with a dashed outline. 
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Figure 25 shows a layout of hexagonal tag 770 using the symbol of Figure 1, with twelve interleaved 2 4 -ary 
(4, k) codewords. Like the hexagonal tag of Figure 8 discussed earlier, the tag of Figure 25 contains six 
codewords, each arranged according to the same layout but each rotated according to a different one of the 
six possible rotations of the tag with respect to its own six- fold rotational symmetry. 

5 

The tag of Figure 25 contains six additional codewords, laid out in the mirror image of the original six 
codewords with respect to the tag's three axes of reflection 772, 774 and 776. As discussed above, this 
allows a codeword to be sampled and decoded independently of the actual rotation and reflection of the tag, 
which allows the rotation and reflection of the tag to be encoded in the codeword(s). 

10 

Each codeword may explicitly indicate its rotation and reflection via a four-bit value, or implicitly via a one- 
bit value indicating whether the codeword corresponds to a pre-determined rotation and reflection or not. 

In Figure 25, each original symbol has a prefix (1-6) indicating which codeword it belongs to, and a suffix 
15 (A-D) indicating its position in the codeword. Each reflected symbol is shown with a bar above its label, as 
well as with a dashed outline. 

Note that the tags of Figure 24 and Figure 25 are not ideally sized for the number of symbols each codeword 
contains, since the data content of each codeword is low once allowance is made for useful redundancy. 
20 However, both serve to illustrate interleaved rotation-symmetric and mirror-symmetric tag designs, and the 
design principles obviously extend to tags of larger size and/or to tags incorporating different error-detecting 
and error-correcting schemes. 

1.2.3 Tag Image Processing and Decoding 
25 Figure 26 shows a tag image processing and decoding process flow. A raw image 802 of the tag pattern is 
acquired (at 800), for example via an image sensor such as a CCD image sensor, CMOS image sensor, or a 
scanning laser and photodiode image sensor. The raw image is then typically enhanced (at 804) to produce an 
enhanced image 806 with improved contrast and more uniform pixel intensities. Image enhancement may 
include global or local range expansion, equalization, and the like. 

30 

The enhanced image 806 is then typically filtered (at 808) to produce a filtered image 810. Image filtering 
may consist of low-pass filtering, with the low-pass filter kernel size tuned to obscure macrodots but to 
preserve targets. The filtering step 808 may include additional filtering (such as edge detection) to enhance 
target features. 

35 

The filtered image 810 is then processed to locate target features (at 812), yielding a set of target points. This 
may consist of a search for target features whose spatial inter-relationship is consistent with the known 
geometry of a tag. Candidate targets may be identified directly from maxima in the filtered image 810, or 
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may the subject of further characterization and matching, such as via their (binary or grayscale) shape 
moments (typically computed from pixels in the enhanced image 806 based on local maxima in the filtered 
image 810), as described in US patent application serial number 09/575,154. 

5 The search typically starts from the center of the field of view. The target points 814 found by the search step 
812 indirectly identify the location of the tag in the three-dimensional space occupied by the image sensor 
and its associated optics. Since the target points 814 are derived from the (binary or grayscale) centroids of 
the targets, they are typically defined to sub-pixel precision. 

10 It may be useful to determine the actual 3D transform of the tag (at 816), and, by extension, the 3D transform 
(or pose) 818 of the sensing device relative to the tag. This may be done analytically, as described in US 
patent application serial number 09/575,154, or using a maximum likelihood estimator (such as least squares 
adjustment) to fit parameter values to the 3D transform given the observed perspective-distorted target points 
(as described in P.R. Wolf and B.A. Dewitt, Elements of Photogrammetry with Applications in GIS, 3rd 

1 5 Edition, McGraw Hill, February 2000, the contents of which are herein incorporated by reference thereto). 
The 3D transform includes the 3D translation of the tag, the 3D orientation (rotation) of the tag, and the focal 
length and viewport scale of the sensing device, thus giving eight parameters to be fitted, or six parameters if 
the focal length and viewport scale are known (e.g. by design or from a calibration step). Each target point 
yields a pair of observation equations, relating an observed coordinate to a known coordinate. If eight 

20 parameters are being fitted, then five or more target points are needed to provide sufficient redundancy to 
allow maximum likelihood estimation. If six parameters are being fitted, then four or more target points are 
needed. If the tag design contains more targets than are rninimally required to allow maximum likelihood 
estimation, then the tag can be recognized and decoded even if up to that many of its targets are damaged 
beyond recognition. 

25 

To allow macrodot values to be sampled accurately, the perspective transform of the tag must be inferred. 
Four of the target points are taken to be the perspective-distorted comers of a rectangle of known size in tag 
space, and the eight-degree-of-freedom perspective transform 822 is inferred (at 820), based on solving the 
well-understood equations relating the four tag-space and image-space point pairs (see Heckbert, P., 
30 Fundamentals of Texture Mapping and Image Warping, Masters Thesis, Dept. of EECS, U. of California at 
Berkeley, Technical Report No. UCB/CSD 89/516, June 1989, the contents of which are herein incorporated 
by reference thereto). The perspective transform may alternatively be derived from the 3D transform 818, if 
available. 

35 The inferred tag-space to image-space perspective transform 822 is used to project (at 824) each known data 
bit position in tag space into image space where the real-valued position is used to bi-linearly (or higher- 
order) interpolate (at 824) the four (or more) relevant adjacent pixels in the enhanced input image 806. The 
resultant macrodot value is compared with a suitable threshold to determine whether it represents a zero bit 
or a one bit. 
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Once the bits of one or more complete codeword have been sampled, the codewords are decoded (at 828) to 
obtain the desired data 830 encoded in the tag. Redundancy in the codeword may be used to detect errors in 
the sampled data, or to correct errors in the sampled data. 

5 

As discussed in US patent application serial number 09/575,154, the obtained tag data 830 may directly or 
indirectly identify the surface region containing the tag and the position of the tag within the region. An 
accurate position of the sensing device relative to the surface region can therefore be derived from the tag 
data 830 and the 3D transform 818 of the sensing device relative to the tag. 

10 

1.2.4 Tag Map 

Decoding a tag results in a region ID, a tag ID, and a tag-relative pen transform. Before the tag ID and the 
tag-relative pen location can be translated into an absolute location within the tagged region, the location of 
the tag within the region must be known. This is given by a tag map, a function which maps each tag ID in a 
1 5 tagged region to a corresponding location. 

A tag map reflects the scheme used to tile the surface region with tags, and this can vary according to surface 
type. When multiple tagged regions share the same tiling scheme and the same tag numbering scheme, they 
can also share the same tag map. 

20 

The tag map for a region must be retrievable via the region ED. Thus, given a region ID, a tag ID and a pen 
transform, the tag map can be retrieved, the tag ID can be translated into an absolute tag location within the 
region, and the tag-relative pen location can be added to the tag location to yield an absolute pen location 
within the region. 

25 

The tag ID may have a structure which assists translation through the tag map. It may, for example, encode 
Cartesian coordinates or polar coordinates, depending on the surface type on which it appears. The tag ED 
structure is dictated by and known to the tag map, and tag IDs associated with different tag maps may 
therefore have different structures. For example, the tag ID may simply encode a pair of x and y coordinates 
30 of the tag, in which case the tag map may simply consist of record of the coordinate precision. If the 
coordinate precision is fixed, then the tag map can be implicit. 

1 .2.5 Tagging Schemes 

Two distinct surface coding schemes are of interest, both of which use the tag structure described earlier in 
35 this section. The preferred coding scheme uses "location- indicating" tags as already discussed. An alternative 
coding scheme uses object-indicating tags. 
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A location-indicating tag contains a tag ID which, when translated through the tag map associated with the 
tagged region, yields a unique tag location within the region. The tag-relative location of the pen is added to 
this tag location to yield the location of the pen within the region. This in turn is used to determine the 
location of the pen relative to a user interface element in the page description associated with the region. 

5 

Not only is the user interface element itself identified, but a location relative to the user interface element is 
identified. Location-indicating tags therefore trivially support the capture of an absolute pen path in the zone 
of a particular user interface element. 

10 An object-indicating tag contains a tag ID which directly identifies a user interface element in the page 
description associated with the region. All the tags in the zone of the user interface element identify the user 
interface element, making them all identical and therefore indistinguishable. 

Object-indicating tags do not, therefore, support the capture of an absolute pen path. They do, however, 
1 5 support the capture of a relative pen path. So long as the position sampling frequency exceeds twice the 
encountered tag frequency, the displacement from one sampled pen position to the next within a stroke can 
be unambiguously determined. 

With either tagging scheme, the tags function in cooperation with associated visual elements on the netpage 
20 as user interactive elements in that a user can interact with the printed page using an appropriate sensing 
device in order for tag data to be read by the sensing device and for an appropriate response to be generated 
in the netpage system. 

The invention may also be said to broadly consist in the parts, elements and features referred to or indicated 
25 herein, individually or collectively, in any or all combinations of two or more of the parts, elements or 
features, and wherein specific integers are mentioned herein which have known equivalents in the art to 
which the invention relates, such known equivalents are deemed to be incorporated herein as if individually 
set forth. 

30 Although the preferred embodiment has been described in detail, it should be understood that various 
changes, substitutions, and alterations can be made by one of ordinary skill in the art without departing from 
the scope of the present invention. 
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